A portion of the disclosure (e.g. Appendix A) of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
This invention relates to a system for providing unique audio augmentation of a physical environment to users. More particularly, the invention is directed to an apparatus and method implementing the transmission of information to the usersxe2x80x94via peripheral, or background, auditory cuesxe2x80x94in response to the physical but implicit or natural action of the users in a particular environment, e.g., the workplace. The system in its preferred form combines three known technologies: active badges, distributed systems, and digital audio delivered via portable wireless headphones.
While the invention is particularly directed to the art of audio augmentation of the physical workplace, and will be thus described with specific reference thereto, it will be appreciated that the invention may have usefulness in other fields and applications.
Considering the richness and variety of activities in the typical workplace, interaction with computers is relatively limited and explicit. Such interaction is primarily limited to typing and mousing into a box while seated at a desk. The dialogue with the computer is explicit. That is, we enter in commands and the computer responds.
Part of the reason that interaction with computers is relatively mundane is that computers are not particularly well designed to match the variety of activities of the typical human being. For example, we walk around, get coffee, retrieve the mail, go to lunch, go to conference rooms and visit the offices of co-workers. Although some computers are now small enough to travel with users, such computers do not take advantage of physical actions.
It would be advantageous to leverage everyday physical activities. For example, an opportune time to provide serendipitous, yet useful, information by way of peripheral audio is when a person is walking down the hallway. If the person is concentrating on their current task, he/she will likely not even notice or attend to the peripheral audio display. If, however, the person is less focused on a particular task, he/she will naturally notice the audio display and perhaps decide to attend to information posted thereon.
Additionally, it would be advantageous if physical actions could guide the information content. For example, a pause at a co-worker""s empty office is an opportune time for the user to hear whether their co-worker has been in the office earlier that day.
Unfortunately, known systems do not provide for these types of interactions with computer systems. Most work in augmented reality systems has focused on augmenting visual information by overlaying a visual image of the environment with additional information, usually presented as text. A common configuration of these systems is a hand-held device that can be pointed at objects in the environment. A video image with overlays is displayed in a small window.
These types of hand-held systems have two primary disadvantages. First, users must actively probe the environment. The everyday pattern of walking through an office does not trigger the delivery of useful information. Second, users only view a representation of the physical world, and cannot continue to interact with the physical world.
Providing auditory cues based on the motion of users in a physical environment has also been explored by researchers and artists, and is currently used for gallery and museum tours. These include a system described by Bederson, et al., xe2x80x9cComputer Augmented Environments: New Places to Learn, Work and Playxe2x80x9d, in Advances in Human Computer Interaction, Vol. 5, Ablex Press. Here, a linear, usually cassette-based audio tour is replaced by a non-linear sensor-based digital audio tour, allowing the visitor to choose their own path through a museum. A commercial version of the Bederson system is believed to be produced under the name Antenna Galley Circle(trademark).
Several disadvantages of this system exist. First, in Bederson""s system, users must carry the digital audio with them, imposing an obvious constraint on the range and generation of audio cues that can be presented. Second, Bederson""s system is unidirectional. It does not send information from a user to the environment such as the identity, location, or history of the particular user.
Other investigations into audio awareness include Hudson, et al., xe2x80x9cElectronic Mail Previews Using Non-Speech Audioxe2x80x9d, CHI ""96 Conference Companion, ACM, pp. 237-238, who demonstrated providing iconic auditory summaries of newly arrived e-mail when a user flashed a colored card while walking by a sensor. This system still required active input from the user and only explored one use of audio in contrast to creating an additional auditory environment that does not require user input.
Explorations in providing awareness data and other forms of serendipitous information illustrate additional possible scenarios in this design space. Ishii et al. ""s xe2x80x9cTangible Bits: Towards Seamless Interfaces Between People, Bits and Atomsxe2x80x9d, in Proc. CHI""97, ACM, March 1997, focuses on surrounding people in their office with a wealth of background awareness cues using light, sound and touch. This system does not follow the user outside of their office and does not provide for the triggering of awareness cues based on the activities of the user.
Gaver et al., xe2x80x9cEffective Sound in Complex Systems: The ARKola Simulationxe2x80x9d, Proc. CHI""91, ACM Press, pp. 85-90, explored using auditory cues in monitoring the state of a mock bottling plant. Pederson et al., xe2x80x9cAROMA: Abstract Representation of Presence Supporting Mutual Awarenessxe2x80x9d, Pro. CHI""97, ACM Press, 51-58, has also explored using awareness cues to support awareness of other people.
Another area of computing that relates generally to electronically monitoring information concerning users and machines, including state and locational or proximity information, is called xe2x80x9cubiquitousxe2x80x9d computing. The ubiquitous computing known, however, does not take advantage of audio cues on the periphery of the perception of humans.
The following U.S. patents commonly owned by the assignee of the present invention generally relating to ubiquitous computing are incorporated herein by reference:
Therefore, it would be advantageous if a system was provided that: 1) transmitted useful information to a user via peripheral audio cues, such transmission being triggered by the passive interaction of the user in, for example, the workplace, 2) allowed the user to continue to interact in the physical environment, physically uninterrupted by the transmission, 3) allowed the user to carry only lightweight communication hardware such as badges and wireless headphones or earphones instead of more constraining devices such as hand held processors or CD players and the like, and 4) accomplished and manipulated bidirectional communication between the user and the system.
It has also been considered to be advantageous to provide a user interface to the audio aura system to allow convenient configuration by a user to suit his/her needs.
The present invention contemplates a new audio augmentation system which achieves the above-referenced advantages, and others, and resolves appurtenant difficulties.
In the parent patent application, U.S. Ser. No. 09/045,447, audio is shown to be used to provide information that lies on the edge of background""awareness. Humans naturally use their sense of hearing to monitor the environment, e.g., hearing someone approaching, hearing someone saying a name, and hearing that a computer""s disk drive is spinning. While in the midst of some conscious action, ears are gathering information that persons may or may not need to comprehend.
Accordingly, audio (primarily non-speech audio) is a natural medium to create a peripheral display in the human mind. A goal of the parent application, U.S. Ser. No. 09/045,447 is thus to leverage these natural abilities and create an interface that enriches the physical world without being distracting to the user.
The U.S. Ser. No. 09/045,447 also describes a system designed to be serendipitous. That is, the information is such that one appreciates it when heard, but does not necessarily rely on it in the same way that one relies on receiving a meeting reminder or an urgent page. The reason for this distinction should be clear. Information that one relies on must penetrate beyond a user""s peripheral perceptions to ensure that it has been perceived. This, of course, does not imply that serendipitous information is not of value. Conversely, many of our actions are guided by the wealth of background information in our environment. Whether we are reminded of something to do, warned of difficulty along a potential path, or simply provided the spark of a new idea, opportunistic use of serendipitous information makes lives more efficient and rich. The goal of the U.S. Ser. No. 09/045,447 is to provide useful, serendipitous information to users by augmenting the environment via audio cues in the workplace.
Thus, in accordance with U.S. Ser. No. 09/045,447, a system and method for providing unique audio augmentation of a physical environment is implemented. An active badge is worn by a user to repeatedly emit a unique infrared signal detected by a low cost network of infrared sensors placed strategically around a workplace. The information from the infrared sensors is collected and combined with other data sources, such as on-line calendars and e-mail cues. Audio cues are triggered by changes in the system (e.g. movement of the user from one room to another) and sent to the user""s wireless headphones.
In accordance with the present invention, a virtual representation of a target area, such as an office, school, home is generated, and includes representation of sensors for the audio aura system. A virtual interface is designated to include the generation of cues to indicate when, through navigation of the target area, a user is within a range to interact with a sensor representation. The visual cue includes an indication of the association between sensors and service routines, and an indication of a capability for user interaction with the sensor representation. Further, the virtual interface connects to the audio aura system via a data link whereby data input by a user through the virtual interface is transmitted to the audio aura system.
Further scope of the applicability of the present invention will become apparent from the detailed description provided below. It should be understood, however, that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit of the scope of the invention will become apparent to those skilled in the art.